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ABSTRACT 

From a population of 130 boys between 7 and 14 years 
of age who had been clinically diagnosed as aggressive, hyperactive, 
or withdrawn, 32, 31, and 32 Ss, respectively, were randomly -selected 
and administered the Quay Behavior Problem Checklist and the Devereux 
Elementary School Behavior Rating Scale during the 1971-1972 academic 
year. A descriptive intercorrelation matrix \.:as generated for the H 
Quay scales and the 14 Devereux scales. Three stepwise discriminant 
analyses were run: Quay scales only, Devereux scales only, and Quay 
and Devereux scales combined. In terms of statistical and practical 
considerations, the 4 Quay subscalea by themselves attained the 
optimal predictive accuracy {65% or 62 out of 95 children corr<5ctly 
identified) . (Avitho^) 
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DISCRIMINATORS OF CLINICALLY DEFIfjED EMOTIONAL MALADJUSTMENT; 
THE PREDICTIVE VALIDITIES OF THE O.UAY AND DEVEREUX SCALES 

Barton B. Proqer, Lester Mann, Paul A. Green, 
Robert J. Bayuk, Jr., & Robert M. Burger 

Montgomery County Intermediate Unit No. 23 
Nor r i s to"wn , Pennsy 1 vani a 

From a pooulation of 130 boys betv;een 7 and 1^ years of aqe who had 
been clinically diagnosed as aggressive, hyperactive, or withdrawn, 32, 31 > 
and 32 S_s , respf-ct i vely , were randomly selected. All S_s were administered the 
Quay Behavior Problem Checklist and the Devereux Elementary School Behavior 
;\atrng Scale during the 1971"1972 academic year. A descriptive Intercor-- 
relation matrix was generated for the 4 Quay scales and the 1^ Devereux 
scales. Three stepwise discriminant analyses were run: (a) Quay scales only, 
(b^) Devereux scales only, and ic) Quay and Devereux scales combined. In 
terms of statistical and practical considerations, the k Quay subscales by 
themselves attained the optimal predictive accuracy (65%, or 62 out of 95 
chiJLdren correctly identified). 
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INTRODUCTION 



In public school operat ion of prggrams for emotionally and soc'.a 



lly 



maladjusted ch i 1 dren ,^ ^ ' 3 ^^^e identification and placement processes raise 
many problems. In attempts to improve diaqnostic procedures, an increasing 



procedures have evolved that involve the teacher who is with the child under 
question riore than any other school official. How accurate are such teacher 
rating scales when compared with independently completed clinfcal diagnoses 
of disturbance made by psychiatrists? This study addresses the question. 
The present investigation is unique in that It pits tv;o commonly used screen- 
ing^scales (the Dcvereux Elementary School Behavior Rating Scale^i? and the 



- The studies contained herein were supported by a research grant to Button- 
wood Farms (a summer camp offering academic and recreational therapy to socially 
and emot fonal ly disturbed children and located on Easton Road, Ottsville, Pennsyl- 
vania 189^2) under the auspices of Montgomery County Intermediate Unit No* 23 
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number of objective personality tests has been the result.^ »5 
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Behavior Problem Checklist 3gainst each other In terms of various val- 
idity considerations relative to a common, outside criterion (individual 
psychiatric diagnoses guided by DSM-TT^) . Most of tfie existing research on 
objective personality scales used in school programs considers the character- 
istics of a single scale in isolation rather than making relative comparisons, 

3, 10, 11, 12, 16, 18, 21, 23, 27, 28, 30, 31 ^ , , 

Yet without comparative research 

as in the present study, few guidelines exist for choosing screeninc devices 

other than subjective critiques.^ 

PROCEDURE 



From a population of about 130 boys between the ages of 8 and 1^ who 
were enrolled in special classes for emotionally disturbed children within the 
suburban Greate* Phlladelnhla Area, 96 S^s v/ere randomly selected to be partic- 
ipants. In particular, stratified random sampling from the population was used 
to ensure that 32 S_s were drav/n from each of the three diagnostic categories of 
aggressive, hyperactive, and withdrav;n. (However, one had to be deleted from 
the^hypera t i ve group because of unforeseen difficulties during the experiment.) 
The original diagnostic classifications in the population had been pre-established* 
in clinical evaluation fashion on the basis of several composite criterfa derived 
from the standard American Psychiatric Association oehavioral categories. The 
three categories were actually condensations of much more detailed diagnoses 
made in accordance with DSM-JM^,^ Collapsing the detailed categories into three 
primary domains v;as guided in Idrge part by previous research.' jhc majority of 
detailed diagnoses were made by fully qualified psychiatrists during December, 1970. 

The investigators arrt deeply grateful to Mrs. Nancy Anderson, Assistant 
Director of Special Education for Intermediate Unit No. 23 in charge of the Learning 
and Adjustment Programs, and to her psychiatric staff. Without their cooperation, 
this controlled investigation would not have been possible. 
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The aggressive group consisted of children v^ho exhibited outer-directed 
motoric behavior, while the hyperactive group contained youngsters who dem- 
onstrated non-directed motoric behavior. The mean age in months for the 
total sample was 121.3. 

The full-scale WtSC, IQs were 101.22 (aggressive), 99.38 (hyperactive, 
and 97.35 withdrawn). The verbal WISC IQ scores were 96. 06 (agg ress I ve) > 
99.28 (hyperacri ve) , and 96.19 (withdrawn) > while the performance IQ scores 
were 106.50 (aggressive), 10^.09 (Hyperactive), and 99.00 (withdrawn). Every 
attempt was made to ensure comparability of CA across diagnostic classifi- 
cations during samplying from the original population so that differences in 
IQ across these three groups would appear to be innate representative ones. 

The data analyzed in the present study came from the summer, 1971 1 phase 
of nultiycor research p'cjcct conducted in a rural surrn'Mer camp setting north 
of the suburban Greater Philadelphia Area. Two rating scale batteries were 
completed for each S^: the 1^ subtests of the Devereux Elementary School Be- 
havior Rating Sca1e^5 and the h subtests of the Quay Behavior Problem Check- 

22 

list. The two batteries were completed by the teachers who were fully av/are 
•» 

of their children's behavioral problems; the teacher ratings on the two batteries 
were completed for the most part between March, 1971 » and the end of the school 
year. Thus, each teacher had more than ample time during the previous months 
of school to become familiar with each child's peculiarities. 

The primary interest of this study was to assess the predictive validities 
of the subtests in each of the two batteries of scales. Three stepwise discrim- 

-The interest of this study v/as striccly in the measurement relationships 
between clinically defined behavioral maladjustment and the two "objective'' 
teacher rating scales. Ha-zever, the multiyear research grant under which this 
study was completed also sou^jht at a later time to offset some of the socially 
inappropriate behaviors exhibited by this particular sample of disturbed children, 
as well as a totally different sample of comparable si2c (for replication's sake). 
The results of the remediation efforts are contained in a large document to be 
available from Educational Resources Information Center (ERIC) and abstracted h) 
a 197^ issue of ERtC's monthly publication, Research in Education. 
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Inant analyses 7» 29 ^^^^ computed using the BM007M program from the Health 
Sciences Computing Facility at UCLA^: (a) Quay scales only, (b^) Devereux scales 
only, and U) Quay and Devereux scales combined. In each of the three analyses, 
the criterion v/as the independently completed, clinically defined behavioral 
status (aggressive, hyperactive, or wi thdrav/n) , while the predictors or dis- 
criminators were the subtests from the scale(s) under scrutiny. The three dis- 
criminant analyses provide one with the basic information needed to assess the 
diagnostic validity of each separate test battery as a whole, as well as to deter 
mine which subtests within each battery do not add significantly to the battery's 
di f feren t ia 1 va 1 i di ty . 

Besides investigating the differential predictive validities of the Quay 
and Devereux seal es , the study obtained a large amount of descriptive informa- 
tion about the nature of the clinically defined behavioral disturbances, filter" 
correlations within and among the ^ Quay subtests and the 1^ Devereux subtests 
v;ere generated, as well as means and standard deviations. 

RESULTS AND DISCUSSION 

*» 

Descriptive Statistics : Because part of the rationale of this study was 
to shed light on behavioral correlates of independent clinical judgments of 
disturbance in children, Table I's results are of considerable interest. 
Of the 150 coefficients presented, 78 are significantly different from 0.000 

insert Table 1 about here 

at the .05 level of confidence, where probabilistically only about 8 coefficients 
should attain such status. Thus, clearly something more than random forces are 
at work among the behavioral ratings of disturbance derived from the Quay and 
Devereux scales. 
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Within the Quay scales, Scale Q-2 (Personality) and Scale Q-3 
(Immaturity) are highly related, as are Q-1 (Conduct) and Q-^ (Socialized 
Del inquency) . 

Among the Oevereux scales. Scale D-1 (Classroom Disturbance) and Scale 
D-9 (Irrelevant-Responsiveness) are of particular interest. In particular, 
D- ( correlates highly v/ith ?-2 (impatience), 0-3 (Disrespect-Defiance), D-^ 
(External Blame), and D-9 itself. Apart from D-1, D-9 Is also highly correlated 
with D-2, D-3, and D-^. Thus, D-1 and D-9 appear to operate In a highly similar 
fash ion. 

A fev; other observations can be made about the intercorrelat ions among 
the Devereux scales^ First, D-3 and D-^ are themselves highly correlated. 
Second, D-6 (External Reliance) and D-8 (Inattentive-Withdrawn) are closely 
associated. Third, 0-^ and D-13 (Quits) share much in common. Finally, D-8 
and D-12 (Unable to Change) are highly related. 

Of particular interest is how the Quay and Devereux scales relate to 
each other apart from their separate internal sets or correlations. One 
striking finding is that Q-^ (Socialized Delinquency) does not seem to relate to 
any of the Devereux scales very v/ell. Among the Devereux scales, neither D-IO 
(Creative Initiative) nor D-ll (Need Closeness to Teacher) relate hfghly to 
any of the Quay scales. 

The descriptive background data in Table 1 also yields what might be con- 
sidered normtng information for highly specific, clinically defined subpopula- 
tions of moladjusted childrer.. One can see clear and distinct differences on 
several of the variable means , mos t notab ly Q-1 , Q-t^ , D-1, D-3i D-^, and D-5. 
Apparently, teachers are qui te valid discerners of cl Inical differenced on 

disturbed chi Idren. Of course, such a finding is hardly stunning in viev/ of 

Id 20 

past research on teacher ratings in academic areas of functioning. 
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Predict I ve Vnl idj ty : Lookinc) at only the Quay scales, (Table 2) Q"3 
( IfTinatur i ty) v/as the best single predictor of diagnostic group membership, 
but only ^1 of 95 children could be correctly identified. Thus Q-^ (Social* 

Insert Table 2 about here 

ized Delinquency) and Q-1 (conduct) had to be added in turn. These 3 scales 
seem to provide the optimally efficient number of correct qroup classifications 
(58 out of 95 or about 6]%). Q-2 (Personality) is of questionable utility In 
this regard. Table 3 provides the two strongest discriminant functions when 
all h Quay scales are included. 

Insert Table 3 about here 

Turning to only the Devereux scales (Table h) ^ one sees that 5 scales 
(D-*^, D-3> and D-11) are needed to produce the opt ional ly efficient number 

of correct group classifications (57 out of 95). 

Insert Table ^ about here 

•* 

Table 5 provides the we'ghts from the two strongest discriminant functions when 

Insert Table 5 about here 

all Dvivereux scales are included. From Table 2 and Table then, one can 
conclude that there is little difference in practical predl ct ion when one com- 
pares 3 of the k Quay scales with 5 of the 1^ Devereux scales. That is, the 
predictive accuracy of the opt imal , practical number of subtests from either 
battery Is about 60Z (57 or 58 out of 95 ch i 1 dren) . However , the analyses 
reflected in Table 2 and Tabled do not tell one what the most effective com- 
bination of both Quay and Oevereux seal es might be . Tab le 6 ref Iccts th Is 
.iv::Sj'tuat ion. . , , 
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Insert Table 6 about here 



From Table 6> one sees that if he would use D-8 (inattentive-Withdrawn), 
D-^ (External Blame), Q-^ (Socialized Delinquency), and Q-1 (Conduct, he would 
achieve a predictive accuracy {60Z) comparable to that reflected above in 
Table 2 and Table k. However, by adding 5 more subtests for a total of 9, 
one can increase predictive accuracy from (>0% to 68%. Table 7 contains the 
weights of the rwo strongest discriminant functions. One should note that 
this optimal combination of 9 subtests contains all ^ of the Quay subtests. 

Insert Table 7 about here 

Considering from Table 2 that all h Quay subtests yield a predictive accuracy 
of (>S% (62 out of 95 children), one would not think It v.'orthv/hi 1 e to administer 
9 selected Quay and Devereux subtests; tfie Quay battery of h scales do as well 
as any other (greater) combination of subtests. Moreover, from a purely prag- 
matic viewpoint, the considerably less amount of scoring activities required 
by the k Quay subtests, or the optimal battery of 9 Quay and Devereux sub- 
scales, indicates that the Quay battery is more than adequate unto Itself to 
discriminate among three clinically defined, behaviorally disturbed groups 
of aggressive, hyperactive, and withdrawn children. 

Much has been wri tten on the p 1 tf a1 Is of cl In leal psych latric diagnostic 
procedures. There are a host of diagnostic classification schemes which 

lend further confusion to this IssucJ^ While one cannot deny the true mis- 
class I f i cat ion dangers inherent In the "profess ional ly made'^ diagnoses by psy- 
chiatrists , 1 t is somewhat refresh Ing to S3e in th 1$ study the fal rly high 
degree of agreement (randing from ^bout 60% to about 68%) between independently 
completed appraisals of the same subjects with behavioral disorders. All 
tannot b6 bad In such a is I tuat ion. 1 1 should al so be noted hero that once 
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again, contrary to stereotypical opinion, teacher ratings have been shown to 
possess a high degree of predictive validity when judged on the basis of more 
refined diagnostic procedures (as used by psychiatrists), ^9»20 

Finally, a few precautions and suggestions for further research are In 
order. First, comparative studies of the relative predictive vaMdittes of 
various screening and diagnostic instruments used in programs for the emotion* 
ally disturbed should be encouraged. To be sure, such comparative studies arc 
methodologically difficult to carry out* At the bare minimum, a comparative 
study needs at least three measures taken on the same subjects: at least two 
measures (the predictors) which are to be pitted against each other and a 
third measure (the crl ter i on) done independently of the predictors. A great 
deal of time, and cooperation is needed on the part of investigators and 
subjects alike- Second, there is a much bigger type of srudy than the present 
one which would shed even more light on the soundness of psychiatric diagnoses. 
One could employ two comparably large samples of ch ? Idren whoreby one sample 
has been predefined as disturbed and the other sample Is presumed normal. The 
InvASttgator would pit professional psychiatric diagnoses against structured 
teacher ratings to see how many false hits and misses and true hits and misses 
result under either method of screening. Clearly, much remains to be done 
in the area of comparative diagnostic validity studies. 
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TAoLE-; 2 

SUMMARY OF D ISCP. lMir.'AMT ANALYSIS 

QUAY SCALE . .. 

Step Variable Entered 3 F Value U-Statlstlc^ Number of 

Nun^ber to Enter ^ - Children 

Correct ly 
Identified 



1 


Q-3 


(Immaturity) 


7.^21 


0.861 




2 


d-k 


(Social ized 
Oe i i nquency) 


10.262 


0.703 




3 


Q-1 


(Conduct) 


7.260 


0.605 


58 


it 


Q-2 


(Persona 1 i ty) 


. 1 . Oi+2 


0.591 


62 



^ KunOcrs refer to duoy factor score noMcs. 

^ The degrees of freedom for the variable entered at Step 1 are 2 ond 92; 
For each step thereafter, the within groups degrees of freedom decrftA<e con- 
tineously by 1 , 

^ The degrees of freedoi^i at each step are 2 and 92. 



TABLf: 3 



OU ^.IMINANT FUNCTION STATISTICS 



QUAY SCALE 


Variable ^ 




Wolqhts 






Fi rst 

Function ° 




Second 
Funct Ion ^ 


Q-l (Conduct) 


-.082 




.158 


(i-2 (Personality) 


- . 054 




.117 


(i-3 (Immaturity) 


.480 




-.058 


Q-'} (Social Ir.od 
Delinquency) 


-,59V 




-.784 



Numbers rerer to Quay t^ctor score names, 

^ The canonical correlation is .568 and the cumulath/e proportion 
of total dispersion is .766; The group means are -.^75 (aggress ive)i 
(hyperactive), and .953 (withdrawn). 

^ ^ The canon-ical correi ation i $ . 356 on the cumul at i ve proport Ion of 
total d i spcrs ion 1 si. 000 . The group means are .457 (aggress ive) ; - .465 
(hypcract i ve) I and -.006 (withdrawn). 



SU MMAIW OF DtSCRl Ji lNANT ANALYSIS 



DEVERruX SCALE 



Stop 
Number 




Vor i at i le 
Entered ^ 


To Enter 


U-btatlstic " 


Number of 
Chi Idren 
Correctly 
Identified 


1 


0-8 


(Inottontlve- 


18.071 


0.718 


50 






Wi tlidrawn) 




2 




(Externa! 


7.172 


0.620 


50 






D 1 anic) 






3 


D-3 


(Oisrcspect- 


1.058 


0.606 


50 






Ocf iance) 








k 


0-1 


(Glass room 


1.158 


0.591 


5h 






Oi sturbance) 






5 


0-n 


(Meed Closeness 


i.2i*7 


0.57^ 


57 






to Tenf tio r*^ 






6 


0-9 


(Irrelcvant- 


1.279 


0.558 


52 






Rosponsivsness) 








7 


0-5 


(Achievement 


1.227 




57 






A nx i t V ^ 






8 


D-12 


(Unable to 


1.0i»0 


0.530 


57 












• 9 : 


0-10 


(C rcat i ve 


0.738 


0.520 


63 






I ni 1 5 at 1 vc) 






to 


0-\k 


(Slov/ Work) 


0.6&8 


0.512 


62 


11 


0-6 


(External 


0.969 


0.500 


61 






ftel lance) 








12 


0-7 


(Cornprehens fon) 


0.277 


0.it97 


65 


13 


0-13 


(Qui ts) 


0.110 


0.i*96 


63 




0-2 


( Inipafc icnce) 


0.000 


0.it96 


63 



^ Numbers refer to Oevercux factor score names 

^ The degrees of freedom for the variable entered at Step 1 arc 2 and 92* 
For each step thereafter , the wi thin groups degrees of freedom decrease con^ 
t i nuous i y by 1 , 



^ The degrees of freedom at each step are 2 and 92, 



TA'ii.i: 5 

BEST COi»Y MAILABLE discrimuiamt ruf.'CTiON statistics 

OEVERCIX SCALE 



Variablo ^ Weights 





F I rst Funct ion ^ 


Second Funct ion *^ 


O-I (Classroo.Ti Disturbance) 


.012 


.277 


0-2 (Irnpottcncc) 


-.001 


.001 


D-3 (Oisrespcct-Oef lance) 


.100 


-.I6i| 


0-^ (Extcrno! Sla.r.e) 


.063 


-.057 


0^5 (Achfevcfnent Anxiety) 


.011 


.151 


0*6 (Extcrnol Reliance) 


.053 


-.056 


D-7 (Comprchons ion) 


-.032 


-.097 


0-8 {Inattentivc-Wl thdrav/h) 


-.11^+ 


-.076 


u-y ^ 1 rre 1 evuiit-Kespons i venoss ; 


- . 140 


-.136 


0-10 (Creative Initiative) 


.105 


.061 


0-11 (Need Closeness to Teacher 


.026 


-.028 


D-12 (Unable to Change) 


-.063 




D-13 (Quits) 


.012 


-.137 


0-1^ (S lov,' i 


-.117 


.183 



^ flunbors refer to Ocvercux factor score norncs, 

^ The Crznonlcol corrolotion is .677 and the cumulot i vc proport ion of totoh 
(il^pcrt-ton is . C^'i V The group n-oons ore , 75^^ (ogg rcss i ve ) , .526 (hyperoctive), . 

^ The conon[c.-iI correlation f s ,292 and the cumulotivc proport Ion of total 
dispersion 1.000. The group r^cons arc .339 (aggressive), - . 395 (hyperact i vo) , 
'an<l'^"/0^3 (withorown). 



• BKTCOi'YAVAIUBU 



QUAY Ano ^ '^J^3!^_}i^i_^f\j^^ 



'u'Eor ^'^^'^^'^S^t^^^^^ ' rvalue ^ U^Statfstlc ^ Number of Chndren 



1 0-8 (Inottcntivc- 

2 0-4 (f:;<tcrnoi Blome) 

3 Qrk (SocialUcd 

Oo I inquoncy) 

4 Q-1 (Conduct) 



5 D-3 (Disrespect- 

Oct' loncc) 
^ O-I ! (uocj C loscncss 

To Teacher) 
7 0-9 (Irrelcvont- 

Responsiveness) 

6 Q-3 (lfr.moturI ty) 

9 Q-2 (Personality) 

10 D-7 (Comprehension) 

11 ; 0-10 (Croctivc 

Initiative) 

12 0-12 (Unable to 

Change 

13 0-6 (Externa) 

Re I iance) : 

14 0-14 (SlovrWork 

15 0-5 (Achiovcniont 

Anxiety) 
.16 0-1 (Ci2Ssro:v;\ 

. i i V U # > i lu u) 

17 D-2 (Impatience) 

18 D-13 (Quits) 



18.071 


0.7J8 


50 


7.172 


0.620 


50 


3.282 


0.578 


52 


^.'+95 


0.525 


57 


2.^495 


0.i497 


58 


U7Bk 




58 


2.023 


QMS 


62 


1.725 


0.438 


62 


1.239 


0.i*26 


65 


0,680 


0,419 




1.033 


0.^08 




o.kys 




67 


0.56^ • 


0.398 


68 


0.508 


0.393 


67 


0.322 


0.390 


67 


0,0o5 


0.389 


67 


0. Qk8 


0.389 


67 


0.026 


0.388 


67 



Nu.Vocrs -oicr to Quoy (i to 4) and Ocvcroux (l to I4) factor score names. 

The degrees of freedom for the variable entered ot Step 1 and 2 and 92 
For^caCh step thcrcoitur, the viltlun groups dc<jrecs of freedom decrease continuously 



BEST COl'Y AVAiLABLt 



pfSCRIMINANT ru s I : LION r^TAT ISTICS 



Variables 



First Function 



Woiqhts 



Socond Fanctton 





(Coiiduct) 


.034 


.256 


(i-2 


(I'ersonal 5 ty) 


.075 


. 068 


Q-3 


( Immatuf-? ty) 


-.2t5 


.060 


Q-li 


(Socialiied Oellnqucncy) 


.428 


-.746 


0-1 


(C I OSS room D i sturbance) 


-.024 


.034 


0-2 


(Impatience) 


.014 


. 000 


0-3 


(Oisrespect-Dof iance) 


.079 


-.226 


0-k 


(Externa! BlaiTio) 


.055 


.009 


D-5 


(Achievcnienc Anxiety) 


-.010 


.057 


0-6 


(External Rel Jance) 


.040 


-.059 


0-7 


(Cor:prohorision) 


-.065 


-.125 


0-8 


(Inattentivo-Withdrawn) 


-.068 


-.120 


0-9 


( 1 r re 1 e va n t - Re s po n s i ve ne s s ) 


-.167 


; : -.072 


OHO 


(Croat fve Initiative) 


,104 


.015 


O-lt 


(iN'cod C loseness t9 Teacher) 


.059 


-.017 


b-12 


(Un.rjie to Chanrje) 


-.009 ■ 


.208 


D-U 


(Quits) 


.029 ' 


-.014 


0-1 


(SlovM-.'ork) 


-,108 


.028 



;:u:-,b.r. refer to Quay (1 lo 4) and Dovoreux (1 to l4) factor score na^nes. 

disporsion'''i^''"7^^'' ^i^'''"^ "'"^ ' °" ''^ cu.mlative proportion of total 

and'!l!;5; (;.iti;d?a;n) ''''' ^^'^ ''''' ^-39ressive). 0.659 (hyperactive), 

:v,I-S'i"T;^"' "^.^''"'^^''^'^ ''^ •'^^^ the cumulative, proportion of total 
^ym ^s 1...... rno .jroL-p moans are 0,603 (oi^^rcss ivo) . -0.b>7 (hypor.Ct ivo) 



